Weight and the Future of Space Flight Hardware Cost Modeling 


Frank A. (Andy) Prince 
Engineering Cost Office 
NASA/Marshall Space Flight Center 


Abstract 

Weight has been used as the primary input variable for cost estimating almost as 
long as there have been parametric cost models. While there are good reasons for using 
weight, serious limitations exist. These limitations have been addressed by multi- 
variable equations and trend analysis in models such as NAFCOM, PRICE, and SEER; 
however, these models have not be able to address the significant time lags that can occur 
between the development of similar space flight hardware systems. These time lags 
make the cost analyst’s job difficult because insufficient data exists to perform trend 
analysis, and the current set of parametric models are not well suited to accommodating 
process improvements in space flight hardware design, development, build and test. As a 
result, people of good faith can have serious disagreement over the cost for new systems. 

To address these shortcomings, new cost modeling approaches are needed. The 
most promising approach is process based (sometimes called activity) costing. 

Developing process based models will require a detailed understanding of the functions 
required to produce space flight hardware combined with innovative approaches to 
estimating the necessary resources. Particularly challenging will be the lack of data at the 
process level. One method for developing a model is to combine notional algorithms 
with a discrete event simulation and model changes to the total cost as perturbations to 
the program are introduced. Despite these challenges, the potential benefits are such that 
efforts should be focused on developing process based cost models. 


Introduction 

Does weight have a future as an input variable in NASA cost estimating 
relationships? Recent concerns raised by NASA managers might lead one to believe that 
weight based parametric models are as outmoded as a Model T on the Washington 
Beltway. An experienced cost analyst may be tempted to dismiss such comments as 
evidence of a lack of understanding of our models and data bases, or as the griping of one 
whose project has been affected by an estimate that is higher than one believes to be the 
case. But should we be so smug as to suggest that the fault lies totally outside of our fine 
profession? Perhaps we should take the time to try to hear the truth behind the 
complaints. Perhaps we should examine our methods and data to see if they are capable 
of addressing the shortcomings highlighted by our critics. Perhaps we should explore the 
development of new cost models that would enable all the stakeholders of a cost estimate 
to come to a consensus based the true cost drivers. 

The purpose of this paper is to perform such an examination of our models and 
methods in light of recent criticisms. The paper begins by examining our current 



parametric models and databases to determine if the issues raised by management are 
supported by the facts. Of particular interest is the value that weight brings to a cost 
model. Once the examination is complete, a new method is proposed that can bridge the 
gap between the cost estimator and management: process based cost models. Finally, a 
simplified development approach is outlined for process based cost models that can 
overcome the limitations in data. 


Weight and Cost Estimating 

Why is weight (or board count, or volume, or some other measure of size) almost 
universally used as a cost model input for aerospace cost models? The answer is very 
simple: it is available and it works. Some of NASA’s earliest cost models used weight as 
the key input parameter. One of the reasons weight was attractive was that it was known 
for both historical data points and new concepts under consideration. The other, and 
more important reason, is that all things being equal, there is a consistent correlation 
between size and cost. This relationship is demonstrated in Exhibit 1 by plotting the total 
cost development cost of all NAFCOM space vehicles (satellites, human rated vehicles, 
and upper stages) against weight. 
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Exhibit 1. Relationship between Weight and Cost of Space Vehicles. 

An important point often lost is that weight is a cost predictor, not a cost driver. 
Exhibit 1 illustrates this point. Notice the relatively low R 2 value and the large absolute 



variance in the data. For example, a 1000 pound space vehicle could cost anywhere 
between $100M and $1000M. Obviously, not all cost drivers are being addressed. 

The traditional approach within NASA to dealing with the variance problem was 
to segment the database into like systems and subsystems. However, experience shows 
that this segmentation introduces problems with the statistical analysis of small data sets 
and, while it reduces the magnitude of variance in the model, it does not reduce the 
relative variance. This is illustrated in Exhibit 2, spacecraft structures total cost and 
weight. The bottom line is that while weight can get us in the ballpark, it doesn’t tell us 
what seat we are in. 


Robotic Earth Orbiting Structures Total Cost 
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Exhibit 2. Spacecraft Structures Total Cost - Weight Relationship. 

Identification and Application of the Real Cost Drivers 

Weight is a great scaling parameter. Bigger things cost more, and when 
everything is equal, historical data points can be used in combination with the appropriate 
scaling relationship to estimate the cost of the new system. However, weight (or any 
other size parameter) cannot totally explain the cost. In fact, many factors go into 
determining the true cost of space flight hardware systems. Some of these cost drivers 
are listed in Exhibit 3. 
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Exhibit 3. Cost Drivers. 

These factors are well recognized by the experienced cost analyst and have been 
captured in various ways by numerous cost models. For example, the NAFCOM model 
uses system design parameters such as output power, design life, storage capacity, and 
power regulation to account for electrical power system design factors. NAFCOM also 
provides input parameters to reflect the management approach drivers identified in 
Exhibit 3. These include manufacturing methods, engineering management, new design, 
risk management, funding availability, integration complexity, and pre-development 
study. When these parameters are correct, the results can be quite impressive, as shown 
in Exhibit 4. 
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Exhibit 4. Example Relationship between Predicted to Actual Cost in NAFCOM. 









Exhibit 4 shows that when a conscientious effort is made to incorporate the true 
drivers of cost into a model, accurate cost models are the result. Given such evidence, it 
would appear that NAFCOM (and PRICE and SEER) have solved all of our estimating 
problems. Unfortunately, the critics of parametric cost models have some legitimate 
complaints. These are: 

• Predicts the past better than the future 

• Limited support for technical design trades 

• Limited support for management and process trades 

• Small, imperfectly understood data sets 

The latter complaint is especially relevant to the problem of accurately estimating 
the cost of new, reusable launch systems. Since there is only one analogous data point, 
the Space Shuttle, trend analysis is impossible. Technical, management, and process 
trades are also difficult to perform with only a single, historical data point. In the case of 
the RLV, the single data point problem is compounded by the age of the data point (the 
Shuttle was developed in the 1970’s). The result is a situation where the credibility of the 
estimate can be questioned. 

A Closer Examination of the Problem 

The heart of any cost estimate is the supporting data. Plentiful and recent data 
that is applicable to the concept being estimated increases the confidence of both the 
analyst and the program manager in the cost. However, when data is neither plentiful nor 
recent, disagreements can arise over the interpretation of the data and it’s applicability to 
the concept under study. This concept is vividly illustrated by Exhibits 5 and 6. 

Exhibit 5 demonstrates the kind of analysis that can be performed when sufficient 
data is available. In this case, a large database of historical small satellite missions has 
enabled The Aerospace Corporation to develop a mission complexity factor. This 
complexity factor can be plotted against cost. By identifying successful missions, a trend 
line can be developed which can be used to determine the likelihood of success or failure. 
Such an analysis, which is based on recent, relevant data, is useful to both cost estimators 
and program managers. 


Total Flight System Cost as Function of Complexity 
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Exhibit 5. Aerospace Corporation Mission Complexity Factor. 


Compare the example in Exhibit 5 with Exhibit 6. Exhibit 6 provides an example 
of the issues faced when estimating the cost when the supporting data set is sparse and 
old. 
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Exhibit 6. Comparison of Shuttle and RLV Development Paradigms. 

The Space Shuttle was developed almost 30 years ago. Since then technologies, 
design software, computer capabilities, and requirements have evolved. In addition, new 
processes in the management of large, high technology programs have been developed 
that may or may not have an impact on the development of the next generation reusable 
launch vehicle. 



















This time lag between the development and production of the only relevant 
historical data point and the new system can be called a temporal/cultural/technology 
gap. The lack of real data to fill in the gap means that the knowledge base used to adjust 
the parametric estimate is theoretical. It also means that consensus will be difficult to 
reach for program managers and cost estimators. The uncertainty engendered in NASA 
management over this very issue and the resulting wide range of cost estimates was one 
of the key factors in the decision to refocus the Space Launch Initiative program away 
from a replacement for the Shuttle and towards the development of an orbital space plane 
(OSP). 

The Next Generation of Cost Models 

Given that the temporal/cultural/technology gap exists, what can be done? Since 
the gap is caused by a limited data set, traditional methods such as adding more technical 
or programmatic input variables are not feasible. Other methods must be pursued. These 
methods must be able to address issues such as improvements in computer technology, 
advancements in design methods, and changes in manufacturing processes. These 
methods must enable the transfer of applicable experience from programs of similar scale 
and purpose. These methods should anticipate the questions and explain why the 
estimated cost is realistic. 

Process based cost models may be the best approach for a tool to bridge the 
temporal/cultural/technology gap. Process based cost models translate spaceflight 
hardware system characteristics into a set of discrete activities. These activities represent 
processes that must be performed to develop and produce the hardware. A typical 
process flow for developing a spacecraft avionics subsystem is shown in Exhibit 7. 





Exhibit 7. Example Process Flow. 


There are several advantages to process cost model. Among these are: 

• Relate cost to work performed 

• Direct linkage to schedule 

• Ability to tie cost to specific functions or activities (like tests) 

• Identification of causal relationships between the cost, schedule, and engineering 

and management decisions 

• Enhance communications between Project Managers, Cost Professionals and 

Technical Disciplines (explain Why, not just How Much) 

• Anticipate cost impacts of design, development, and manufacturing trends 

• Provide engineering cost support beyond early formulation 

These advantages directly address the temporal/cultural/technology gap issues. 

By focusing on the relationship between the process and the product, how the system is 
being developed can be directly estimated. 

The potential for revolutionizing the cost analysis profession is enormous. 

Process cost estimating combined with discrete event simulation will enable realistic 
simulations of an entire program life cycle. Management decisions can be tested for 
affects on cost and schedule while the program is still in the formulation phase. 
Development assumptions can be tested for robustness. Engineers can see how 
performance characteristics affect O&S cost. Cost estimates can be easily justified and 
understood because the results can be traced directly to the work performed. 

Development of a Process Based Model for OSP 

Development of a process-based model for the OSP program is a doable but 
challenging task. The best analog for developing the model is the Space Shuttle Orbiter. 
The processes followed in developing and producing the orbiter and their duration are 
known. Significant documentation exists and experienced aerospace engineers can 
provide a wealth of first hand information. 

The difficultly in the development of a process based cost model for the OSP 
program will lie in determining the labor hours of the processes. Labor hours are used 
instead of cost to provide a more accurate reflection of the work performed. Existing 
orbiter labor hours documented by work breakdown structure (WBS) element, not by 
process. Reanalyzing the historical labor hour data to fit a process paradigm will require 
innovative and defendable approaches to allocate WBS labor hour data to the processes. 
Once the notional labor hour allocation is performed, the next step will be to determine 
the proper inputs for estimating the labor hours and schedule duration of each process. 

To determine the proper inputs and responses of the model, heavy use must be 
made of expert opinion. Discussions will be held with subsystem design engineers, 
supervising engineers, system engineers, and program managers to uncover the inputs 
that drive labor hours and schedule duration. Expert opinion will also be used to develop 
notional relationships between the inputs and the changes in labor hours and schedule. 

The best approach to testing the notional process model may be to use discrete 
event simulation to calculate the probable outcome given a set of input data. The 



methodology for developing a subsystem process simulation would rely on the following 
information: 

• Subsystem level labor hours 

• Shuttle Orbiter program documentation 

• Basic notional process flow model 

• Discrete event simulation of the process flow 

Once these four items are in hand, the discrete event simulation can be calibrated 
against actual orbiter program experience. The calibration exercise will follow the four- 
step process outlined below: 

• Adjust the simulation to reflect the orbiter program as envisioned at authority to 
proceed (schedule, labor phasing, etc.). 

• Modify the simulation to reflect challenges faced by the orbiter program, such as 
funding shortfalls, technical failures, and requirements changes. 

• Calibrate the simulation to yield the same labor hours and schedule changes that 
resulted from the orbiter program challenges. 

• Document the changes made to the simulation. 

The use of discrete event simulation will enable the uncertainty in the notional 
labor hour allocations and models to be bounded. If the model gives reasonable results 
that agree with historical experience, the determination can be made that the model is 
adequate. The model can now be used to estimate the effects of improved management, 
engineering, and manufacturing processes. Credibility in these estimates can be 
established by capturing the experience from relevant programs into the model and using 
the model to verify the results. The model will also provide the template for how the 
OSP program must be managed for cost savings due to improved process can be realized. 

Conclusion 

The development of a process based cost model is a daunting challenge. 

However, if our profession is to move beyond estimating the hardware system to 
estimating how the system is developed, this challenge must be faced and overcome. 
Project managers, designers, and manufacturing engineers are constantly seeking ways to 
do things better. The cost analysis community must step up support these professionals 
by providing the cost impacts of their attempts to find better and more cost effective 
solutions to developing spaceflight hardware. 
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